Migrating UserVoice Ideas To GitHub

We are all about GitHub on the docs.microsoft.com team. We host documentation there and just recently we launched content feedback that’s storing comments in GitHub issues as well. Today, we moved all site feedback to GitHub as well.

(Source: Pixabay)

Prior to the move, all our site suggestions and feedback were on UserVoice, and as we kept consolidating the feedback triage locations, site feedback was next in line. But how do we accomplish this?

Manually moving things over was not exactly my style, so I thought I would script it. There were two key things in place that helped me - the UserVoice Python SDK and PyGitHub. You can get the same script that I used by looking at my tools repo.

Following the script, first thing we need to define all the required automation credentials. That is - the API keys.

# UserVoice account ID. This is part of the URL, e.g. for msdocs.uservoice.com, this would be msdocs.
USERVOICE_ACCOUNT_ID = ''
USERVOICE_API_KEY = ''
USERVOICE_API_SECRET = ''
USERVOICE_SSO_KEY = ''
USERVOICE_CALLBACK_URL = 'http://docs.microsoft.com/'

GITHUB_TARGET_REPO = 'MicrosoftDocs/feedback'
GITHUB_PERSONAL_ACCESS_TOKEN = ''

For UserVoice, you can get that by going to https://{your_account}.uservoice.com/admin/settings/api:

If you are not using SSO-based authentication, you can safely skip USERVOICE_SSO_KEY.

Once you have the API keys, you can specify them in the string variables I showed above - USERVOICE_API_KEY and USERVOICE_API_SECRET.

For GitHub, you can get a standard Personal Access Token (PAT) via the developer console - https://github.com/settings/tokens. Make sure that it has repo-level access to be able to create new issues.

Last but not least, GITHUB_TARGET_REPO should be set to the GitHub repo ID where the issues will be created.

Now, we need to jump to some extra cleanup pre-processing - it so happens that some suggestions contain profanities, and I wanted to make sure that we avoid posting those on GitHub once we move the issues over. After a bit of research, I’ve stumbled across this answer on Stack Overflow that showed how to implement a rudimentary filtering mechanism that works well enough for my scenario.

That’s how purifier.py came to life - we can import a new ProfanitiesFilter class:

f = ProfanitiesFilter([''], replacements="*") 
f.inside_words = True

Now we can initialize both the UserVoice and GitHub clients, and start getting the list of suggestions that were posted:

# GitHub Client
g = Github(GITHUB_PERSONAL_ACCESS_TOKEN)

# UserVoice Client
client = uservoice.Client(USERVOICE_ACCOUNT_ID, USERVOICE_API_KEY, USERVOICE_API_SECRET, callback=USERVOICE_CALLBACK_URL)

suggestions = client.get_collection("/api/v1/suggestions?sort=newest")

# Loads the first page (at most 100 records) of suggestions and reads the count.
print ("Total suggestions: " + str(len(suggestions)))

When getting the list of suggestions, I thought I would get a mirror array that contains only ideas that are still open - that means nothing that’s marked as completed or declined:

ideas_to_migrate = []

print ('Collecting suggestions...')

# Loop through suggestions and figure out which ones need to be migrated.
for suggestion in suggestions:
    if suggestion['status']:
        status_type = suggestion['status']['name']
        if status_type.lower() != 'completed' and status_type.lower() != 'declined':
            ideas_to_migrate.append(suggestion)
    else:
        ideas_to_migrate.append(suggestion)

Last but not least, once the ideas are ready - moving them over to GitHub is relatively easy with the help of create_issue - depending on the existing labels you have in your GitHub repo, you can map UserVoice statuses to new GitHub issue labels.

In addition, I’ve also added an attribution string that will be representative of who originally opened the idea - that will be appended to the issue text:

migration_count = str(len(ideas_to_migrate))
print ("Number of suggestions to migrate: " + migration_count)

target_repo = g.get_repo(GITHUB_TARGET_REPO)

counter = 0
print ('Kicking off migration to GitHub...')
for idea in ideas_to_migrate:
    counter += 1
    print ('Migrating idea ' + str(counter) + ' of ' + migration_count + "...")

    idea_text = '_No details provided._'

    if idea['text']:
        idea_text = f.clean(idea['text'])

    # String that defines the attribution block of the issue.
    attribution_string = '\n\n----------\n⚠ Idea migrated from UserVoice\n\n' + '**Created By:** ' + idea['creator']['name'] + '\n**Created On:** ' + idea['created_at'] + '\n**Votes at Migration:** ' + str(idea['vote_count']) + '\n**Supporters at Migration:** ' + str(idea['supporters_count'])

    # Define labels
    labels = []
    if idea['status']:
        status_type = idea['status']['name']
        if status_type.lower() == 'under review' or status_type.lower() == 'planned':
            labels.append('triaged')
        elif status_type.lower() == 'started':
            labels.append('in-progress')


    target_repo.create_issue(f.clean(idea['title']), idea_text + attribution_string, labels=labels)

And there we have it!

30 minutes of coding saving hours of manual work - just like automation was supposed to work!